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^ (54) Title: IMAGING APPARATUS 

(57) Abstract: Imaging apparatus comprising two imaging units (12) for producing images of respective scenes, each imaging unit 
JJ^ comprising optical means for gathering light over a wide sector and directing it to image sensing means (14). The optical means 

of the imaging units can be back-to-back so as to encompass the imaging sensing means (14) and each unit can include a convex 

reflector (16) for reflecting light torn a panoramic scene onto a planar reflector (18) which reflects light through a port or ports (24) 
^ in the convex reflector (16) onto a CCD array (14) within the convex reflector. Other reflecting systems include convex, concave 

and planar reflecting surfaces. Also disclosed are arrangements for reducing back-workings in the back-to-back fisheye lenses; a 
O ^CD with a curved image sensing surface; light shielding means; a method of generating position signals for navigating round a 

panoramic scene; and a device including a segmental field of view and an annular field of view which are combined to provide a 
^ panospherical image, the image output is processed to form a composite image in a flat plane. 
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IMAGING APPARATUS 

S This invendon idates to imag^ appaiHtus, in particular to* qiparatas for producing an 
image ofapanoranuc or omniidirectional scene. Theinv^oncanbeused^fbr^anpl^ 
to provide a relatively simple and cost-effective survdllance q^st^ for monitoring a 
large zone from a single vantage point However, it can also be used wherev^ there 
is a need to collect imagp iiq>ut infonnation over a wide sector, especially o 

10 360 deg., in any of three orthogonal planes, e.g. by means of imaging devices such as 
a video cam^ or CCD array (or in some cases by film). For e?aunple, a large zone can 
be monitored on a single screen using only one stationary imaging apparatus. 
Conventional survdUance systems require many video cameras to be distributed 
throughout the zone to be monitored and these are often panned and tilted to observe 

IS the ^\^le scene. Alternativd^, the invention can be ^plied in broadcast or transmission 
systems, such as television. It can also be mounted on a land, sea, or air vdiicle to 
provide an all-round fidd of view. The resultant images may be transmitted on the 
Internet or ^de^ or local, area netwoik, or via satellite, cable or terrestrial digital or 
analogue video systems. The invention can also be sqpplied in a 360 by 360 deg. 

20 interactive film or video system. 

US-A-5760826 disdoses an omnidirectional imaging ^aratus conipising paraboloid 
reflectors arranged dther singly for reflecting an image of a hemispherical scene onto 
a camera, or doubly, i.e. back-to-back, to reflect a spherical image onto a camera. 

25 However, the disadvantages are that the cameras are located "outside" the paraboloid 
reflectors, i.e. spaced apart from and looking at each reflector, and they are widely 
spaced apart since they are airanged on the longitudinal axis passing through each 
reflector. This makes the system is difficult to instaU and unsuited for locations whe^ 
an unobtrusive^ compact and enclosed unit is needed. Attempts to make a portable unit 

3 0 result in having a camera supported on an extended post passing through the axis of the 
paraboloid mirror, the camera being bcated r^otefy fi^om the mirror so as not to blodc 
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the "overhead" sector of the panoramic view. This results in a cumbersome elongated 
structure, especially when two paraboloid mirrors are joined back-to-back and 
respective cameras are mounted on long posts whidi extend in opposite directions away 
from the mirrors. 

5 

One aspect of the invention solves the latter problem by providing imaging apparatus 
which comprises at least two imagyog units, each unit indudiog optical means with a 
wide field of view, the apparatus also including image senang means for recdving an 
linage from the optical ineans so as to provide a coire^ndiiig oxilpu^ 
10 being adjacently positioned (a) so as to encompass eith^ the image sming means, or 
iiuage divesting means, which diverts images to the image smffiog means, and (b) so that 
the output can be used to form a single composite image in the same plane. 

The apparatus would normally include processing means for processing the output of 
IS the image sensing means so as to form a single composite image in the same plane and 
such processing means can be of known construction and operation (as e>q)lained 
herein). The fields of view of the respective optical means are generally directed 
adjacently so that the images captured by each have a common adjacent boundary, e.g. 
the equatorial boundary in. back to back hemispheres^ and or the adjoining edges of 
20 planar &cets, vA^e^y a single continuous image of the ^ole, or part of the panoramic 
scene can be fi>rmed on said image plane. 



An advantage of the invention is that it can make the structure of the omnidirectional 
25 imaging apparatus more compact and robust. In some embodiments of the invention, 
for example where back to back convex reflectors are used, the imaging units or 
reflectors form a shell, which houses the image sensing means, such as a CCD, within 
an "inner r^on". In another embodiment, for example where back to back fisheye 
lens systems are used, each fisheye 1^ is effectively sq}arated into two parts, whereby 
30 a part of each fisheye lens system is outdde the inner re^on enconq>assed by the back 
to back units; the unage deflecting means is in the ""inner r^on"; and light is reflected 
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onto the other part of each lens systems oiitside the inner regLoa An advantage is that 
the length (along the conunon leais axis) between the outermost regions of the back to 
back fisheye lens systems is greatly reduced. (Pisheye lens systatns generally have many 
different lenses arranged in a sequeaice in the system. These are known as the ''back 
S workings^', and they can be lengd^ conq)aied to ordinary lenses and back to back units 
are cumbersome.) 

Preferably, the imaging units are arraqged back-to-badc on a common asds of symmetry 
\^ereby the imaging imits provide a housing for the image seaosing means, and hence 
10 the advantages of protecting the imaging sensing means and a more robust structure 
whilst also providing for 360 x 360 omnidirectional viewing. 

Where lenses are used to gather Ught from reflecting surfeces, these can be telecentric, 
sudi as telecentric video lenses Cmchiding the LS2-271 and others marketed by Edmund 
IS Industdal Optics) which provide constant magnification over a defined working distance 
range and help to reduce or eliminate perspective distortion and magnification oror in 
obsai^ 3D objects. 

Some embodiments of the invention use reflecting systems to cq)ture images from a 
20 panoramic scene, eadbi 'vmffng unit induding first and second reflecting means, the first 
being convex for rdflecting an image from a panoramic scene onto the second which, in 
turn reflects the image onto the imaging sensing means. For example, the first reflecting 
means can be hemispherical, parabolic, hyj^erbolic, ellipsoidal, or of a polygonal type 
where the polygon inchides a plurality of planar or curved reflecting facets surrounding 
25 a central axis through the convex reflecting means. These can be whole or frusto-convex 
(truncated) sections. The second reflector can be planar, concave or comes, and any of 
the hemispherical, parabolic, hypeibolic, dlipsoidal shapes. Where the first and second 
reflecting means have a curvature, these are in confiscal rdationships. A particularly 
usefiil reflective system is disclosed in US4012126 (Rosendabl). 

30 

In one example (a) of a polygonal structurei, a conv@c pyramidal reflecting system has 
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an aperture or light port in its apcK which is aligaed with the cmtre of the base of the 
pyramid on the axis of symmetry. It also has aplanar reflector for reflecting light, from 
each facet or side of the pyramid, through the light port onto the image sensing means. 
In another example (b), each of the &cets of a convex pyramidal reflecting system has 
S an aperture, pref^iably at a mid-point, and seising means located beneath the aperture 
so as to view the reflection of the respective planar &cet in the planar mirror. In 
another example (c), a system for gathering light is located beneath an aqperture in the 
&cet and the lig^ is conducted, via a lig^it guide, sudi as an optic fibre, to the sensing 
means. Aphiralily of su(±i fight guides can be connected to a manifold for composing 

10 respective parts of the panoramic scene Q.e. the images in the &c^) directly onto the 
image plane of a camora or CCD. Appropriate lenses can be used for focussing as luay 
be necessary. In a further example (d), a system for gathering light is located directly 
opposite the facet and this light is conducted, via a light guide, to the sensmg means. 
A plurality of such light guides and a manifold for composing respective parts of the 

15 panoramic scene (ie. the images in the facets) direct!^ onto the image plane can be used 
as before. In yet another example (e), the convex reflector comprises a (first) set of 
fiicets, which are sides of a polygonal convK reflector having an axis of symmetry, 
wheare the &cets symm^cally surround the central axis so as to reflect respective parts 
of the panoraimc sceoei, the second reflecting means includes a second set of &cets for 

20 reflecting li^ through a lig^ port in the apex ofthe convex reflector, andthesecond 
set of &cets are arranged to reflect only the light which is incident on them firom 
respective fecets in the first set, whereby the image ssosing means separately and 
respectively recdve light firom those parts of the panoramic scene reflected in the first 
set of facets (of the convex reflector). 

25 

In any of these examples, light fi:om the panoramic scene is r^ected by the planar facets 
of the convex reflector (i) either onto the planar reflector, whereby each sensing means 
views a respective part of the panoramic scene seen by the corresponding &cet, (ii) or 
onto the sen^ng means, such as a CCD, or onto a light gathering system, such as an 
30 optic fibre leading to a CCD. This means that only the li^fix)m a particular &cetwi^ 
be incident on a respective part of a common im aging sur&ce (CCD), or on a respective 
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imaging surface of one of several CCDs. This arrangement is important for composing 
the composite image on the image senang means, because it is not gen^ully possible to 
image the reflection in all &cets by just looking down on a pyramid (looking at the 
reflection in all the &cets) with one camera. light baffles may be arranged between 
5 ai^aceot reflective ficete, which baffles cxteaod radially outwardi^inline with the comesrs 
of the pyramid so as not obscure the scene, for further separation of adjacmt sectors 
(e.g. to avoid spurious reflections), ^y such exanq)le5 can be used as a single imaging 
unit for 180 x 360 viewing, or with two back-to-back units for 360 x 360 viewing. 

10 The first and second reflecting sui&ces canbeindependentlsr or mutually adjustable to 
enable the panoramic scene to be composed correctly. For example, the distance 
between the back-to-back units is adjustable so as to adjust the spacing of the first or 
convex reflectors, as well as the distance between the first (convex) and second (planar) 
reflector in each imit The first and or second reflectors can also be mounted so that 

IS each or both are independently, or sinaultaneously pivotable about an axis, tiltable, or 
bodily displaceable. The image sensing means (camera) can also have zoom, and/or be 
mounted in gimbals. All of these adjustments can be respectively and selectively used 
for focussing, scaling, iniagepositbning, whereby the required part or panorama of the 
scene can be correctly imag^ These adjustments can be manual or automatic to adjust 

20 fi3r dififermt scenes or applications. 

Where parts of a panoramic scene are imaged by planar &cets (such as the sides of a 
pyramidal structure), the reflectmg system is adjusted so that each adjacent facet sees 
respective adjacent sectors of the panoramic scene, whereby parts of the sc^e can be 

25 joined or seamed together in the final composite image. When viewing distant objects, 
it is better to use more rather than less fecets, e.g. between 6-10 facets in each unit or 
pyramid (for 180 x 360 viewing), because these provide for better overlap of images at 
the comers of the pyramid, Le between adjacent &cets, for joming images of adjacent 
sectors. Itissuiprisingwhatcanbeseenintheadjacmtfacetsof a pyramid with say 

30 6 sides^ since the panoramic hnage spans the boundary (or comer) between the &cets 
without losing too mudi image information. The rdative positions of reflecting surfaces 
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and the focussing of the image sensing means is adjusted so that the overlap between 
adjacent facet images is optimum for the scene under observation. The angle of slope 
of the &cets is generally 45 deg., but it can be different or made variable. 

S In the case of ludng a 4 sided pyramidal structure in one imaging imit (or 8 sided with 
bade to bade units forming an octahedron), this is not suitable for viewing objects at a 
distance, but it can be used with near objects. For example, it can be used in pipe 
inspection, where the inner pipe wall is dose to the reflectn:^ surfeces and the apex of 
the pyramid (or apices of the pyramids) travd along on the lon^dinal a^ds of the p^e. 

10 Wttk back to back units, the forward lookiQg sides of the pyramid would then reflect 
images of the nearby inner pipe wall for optical inspection, while the rearwardly feeing 
sides can image the same areas a little later, e.g. for inspection by IR, UV or X-ray. 
Pyramids with more sides can be used for pipes of bi^er diameter. Generally speaking, 
convex parabolic reflectors and fisheye lenses are dmple to use, but introduce spherical 

15 distortions which must be compensated by, for example, transformations of distal 
signals which effectivdy develop the sph^cal images into cylindrical images, which are 
then unrolled into adj acent flat sectors that form a continuous image on the same plane. 
This can involve bulky storage and processing of image data signals to map panoramic 
sfphericalunagemformation onto a flat plane. Such mapping is known in the art, for 

20 example, from 'Digital Image Warping"' by George Wolberg, IEEE Computer Sodety 
Press ISBN 0-8186-8944-7; "Space Image Processing"' Sanchez and Canton, ISBN 
08493 3113-7; '*Making environment maps from fisheye photographs" by Ken 
Turkowski, July 1999, which refers to a paper published in 1986 by Ned Qreme relating 
to environmental mapping ( Environment Mapping and Other Applications of World 

25 Projections, in lEE Computer Graphics and Applications, 1986, vol.6, no. 1 1, pp21-29). 
Such solutions rely heavily on complex processing of digital video signals to 
con:4)ensate for optical spherical distortions which are severe at the image boundary. 

Another known tedinique is disclosed in US 6002430 (McCall & Martin), whidi 
mentions back-to-back lenses for cqpturing "hemisphecical images" from a sc^e, 

30 ^'storing" these images (by which is meant dther digitally storing signals or capturing 
images on film), and uamg a "converter" whidi identifies, joms and smoothes the edges 
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(or seams) of the "hemispherical images". Such a "converter" can employ "automatic 
image processing" using "digital processing by a computer where images are altered 
automatically" for combining the two images together. Such prior art techniques can 
be used with embodiments of the invention which employ e.g. fisheye lenses and 
5 parabdoid i^ectois. However, with the advmtofrapid data rates that can be achieved 
on current state of the art conqputers, suchprocesang can significantly add to the cost 
and complexity of the imaging apparatus. Moreover, withincreaaixig demaiKls for even 
lugiher resohition CCDs, for example, when using CCDs, such as Philips or Loran 
Fakchild chips, where say a pair of 18 megabyte dups could image something in the 
10 order of 20x10^ pixels and vAim a fiame rate of 10 fiames/isecond would geaierate data 
in the order of 400 megabytes/second, the comparatively slow execution of complex 
mapping routines would not generate image information at a &st enough rate without 
using exceptionally fast and expensive computers. 

IS In the embodiments of the present invention which use> for example, planar reflecting 
&cets in a pyramidal convex reflector, the anK>unt of processing of digital imaging 
signals is substantially reduced, because no ^unilar conq)lex techniques are required for 
the amount of image vtrarping needed with fisheye lenses, since parts of the image are 
reflected in flat reflecting surfiices, which do not introduce q)hmcal and other 

20 distortions usually assodated with curved reflecting sur&ces. This is a particular 
advantage ova: parabolic reflecting and fisheye 1^ systems. If some slight distortion 
is introduced, e.g. due to perspective, or due wide angle lenses dose to the fecets, this 
can be conq[)ensated fiur more easily than by spherical image transfonnation techniques. 
In other words, curved reflecting surfaces ofler lower optical quality than planar 

25 reflecting surfaces. 

The imaging units each have respective image sensing means or use a common image 
sensor, which can be for example, a ^ddeo camera, a CCD imaging system with 
appropriate lenses, or any other means for producing an output. This could be film in 
30 some cases, but typically an optical image of the scene is focussed onto a device which 
generates m-aaBiBg image of the scene whidi is then converted into digital signals that 
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are processed in order to cany out any necessary warping or image transfonnation for 
composing the final image onto a flat viewing plane. The imaging unit or imits which 
capture the image of the scene optically, can be linked to the imaging sensing means 
either by connection (e.g. conductors), or by transmission/reception systems operating 
S by KF, IR, Bluetooth, Fireware ^sterns, whidi liable (e.g.) image signals to be 
transmitted to a remote point ^^e they are processed for displaying the composite 
image. This &dlitates installation, suu:e the opticd 

be positioned on, or moved around a site, without conc^ for cabling to a remote 
viewiiig station. 

10 

With gmeral regard to the use of lenses, any suitable prescriptions or groi^ings can be 
used for standard, tdecentric, fisheye, supplementary or relay lens systems as will be 
known to those skilled in the art. The same conmients apply to the shapes of spherical 
surfaces, such as hyperbolic, ellipsoidal, parabolic, concave, convex and to confocal 

IS relationships between focussing reflectors that are on the same optical paths. Also, the 
shapes, sizes and spacing of optical elements, including the angles of planar pjrramidal 
sides can be suitably designed according to known optical principles, because the 
illustrations herein are only schemiatic and are not to scale, and fiuther reference can be 
made to works such as ""Modem Optical Engineering" referenced herdn for fiirther 

20 details of construction of the components used in particular arrangements. 

Where means are provided for moving the second reflector relative to the first or convex 
reflector, this can be used to adjust the size of the scene imaged by the imaging unit. 
For example, by moving the second reflector closer to the first reflector, the size of the 
25 scene imaged by the imaging imit is reduced. It can also be used to adjust magnification, 
for example, to focus on a particular part of the scene, for example, to aid identification 
of a person appearing in the scene. 

The imaging apparatus can be used for surveillancei, since the omnidirectional optics can 
30 be located on a n:iast to observe a panoramic scene. It can be protected by annoured 
glass to prevent pl^acal damage and it can also indude protective means to shut down. 
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or to shield the sensitive areas of CCD's in the event of attetnpts to disable the 
apparatus, for example, by shining las^ beams, or other high powered beams, onto the 
mirror system with the aim of bumii^ out the camera. Such protective means can 
mchide, for example;, overlaid transparent panels vMck normalfy adnoit li^ to the CCD 
5 arrays but \^ch are responsive to a detection sagnal so as to tmnjanit red (R), greeaa (G) 
and blue (B) light respective!^. When all three RGS "fitters" are turned on, no 
significant light wiU then reach the CCD arrays. Other kinds of li^ filters or 
defocusangorimagebluningcanbeusedtoasinula^ aKerrcell 
can be used which responds to a detection sagnal so as to go dark or change its 

10 polarisation properties. The protected areas will also be protected if they are blurred 
or defocused. An independent seaosor can be iiK;luded for detecting incident light above 
a given brightness threshold, so that the CCD is protected against damage while the 
threshold is exceeded. The sensor then removes or deactivates the shield, when the 
brightness of the incident light Ms below the threshold. Detection signals alternatively 

IS or also can be g^erated by causing the processing means to respond when a group of 
pixel values exceeds a threshold brightness so as to generate the camera shutdown or 
protection signal. 

In other exainples of the iiEvention, instead of using a "'reflectirig'' s^^ 
20 unit inchides a wide angle 'lens'' system for focussing an omnidirectional image firom 
a scene onto the image sensing means. For example, each imaging unit indudes a 
fisheye lens system. In the latter case, the second reflector and unage focusing system 
are unnecessary. A further advantage is that the back-to-back lens systems have no bUnd 
spots on their central axes, which are otherwise present in some reflecting systems 
25 where light is reflected fi-om an upper (or lower) mirror into the space withm the convex 
reflector. 

The image sensing means, such as a solid state CCD, can have a spherically curved 
image recdving surface, which is shaped optically (e.g. concave or convex) to reduce 
30 the spherical distortion introduced by curved reflectors or, e.g. fisheye lenses. For 
example, the light can be directed more at the normal than at an ang^e to the sensing 
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surfece. This fecilitates trajosfonning data relating to the spherical image into data 
relating to an image in a flat plane. Also the shape of the CCD can be circular to 
correspond with the optical output of the lais (or convex reflector). This maximises the 
resohition output and avoids processing regions of a rectangular CCD on which a 
S drcular image is otherwise focused. This makes more efficiCTt use of pixels on the 
image surfiice. 

When using back-to-back convex reflectors or wide angle lenses with means for 
diverting radiation fo^m an intermediate zone between them, the sen^ means 

10 can be remote from the volume enclosed by back-to-back reflectors or l^es^ so that 
any heat generated by the sen^ng means can be moie easily dissipated. The means for 
diverting radiation from the intermediate zone between back-to-back reflectors or wide 
angle lenses can be any optical device that can turn radiation through a suflBcient angle 
so that it emerges from between reflectors or lenses and is thereby received by the 

IS sensing means which is remoteJy positioned from the optical means. Suitable diverters 
are mirrors and refractors. 

WUdiever type of imaging unit is used, it can have a transparent housing or solid optic, 
preferably shaped symmetrically with respect to the convex reflector to avoid 

20 stigmatism, coma and other optical distortions (but could be cylindrical) to provide 
environmental protection Two symmetrical imaging umts back to hdxk protect the 
enclosed image sensing means. The housing can be rotated to drive off rain by 
centri&gal force, or it can have any suitable wipers which rotate around the housing, or 
about which the housmg rotates. Also heated air systems can be used to pressurise the 

25 housing to exclude dust and to maintain the optics at the same temperature. 

According to another aspect of the invention, an imaging method comprises the steps 
of: 



30 



providing an omnidirectional image of a scene; 
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deriving image signals front said image; 

transforming the image signals into image signal data having three degrees of 
spatial orientation; 

5 

generating position related signals referenced to an artificial horizon; and 

using the position rdated ^gnals to select the image signal data so as to generate 
a display of at least apart of the image of the scene, so that pitch, roll and yaw 
10 of the sensor is transformed into equivalent movemmts of the displayed image. 

This m^od provides the advantage that nM>vemeaat of the portion sensor will cause the 
relevant part of a panoramic scene to be tracked and imaged and this tracking will 
automatically continue to match position related signals. For eTcample^ if the position 
15 sensor were mounted on headgear, simply turning the head and/or tilting the head 
forward and back or from dde to side wiQ provide the pitdi, roll and yaw related signals 
which will cause the part of the panoramic scene to be displayed. . 

In a preferred embodimmt, the digital signals rdating to omnidireclional images can be 
20 stored in memory so that they are available around the whole omnidirectional scene, 
whereby the stored signals of any part of the scene are immediately available for 
processing to remove any image distortions^ and to reproduce a display of that part. 

Generally speaking, whilst the imaging units are preferably arranged to provide a 
25 hemispherical, or panospherical fields of view of the surrounding scene, an **all around" 
view may not, in practice, include viewable parts of the scene which are severely 
distorted, or obscured by supporting stmcture, or wluch correspond to blinds^ 
mirror structures described above. In some cases, peripheral boundaries of the mirror 
or lens systan can introduce mayinium distortion which may not be entirely flattened 
30 into a single continuous image, but this image could be flattened acceptably enou^ or 
ignored, where the rest of the field of view is of nugor importance. Some compromise 
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in image quality may be required, for exBwplty with the plane mirror arrangemmts (such 
as the octahedral housing), but these are less e)q)^Qsive than the curved mirrors and 
fisheye lenses. 

5 Generally speaking, the image sensing nieans produces a digital video sagoai of tte 
im^ge produced by the wide angle optical system and this signal is processed to remove 
or to substantial^ reduce distortion caused by the reflecting Tlieimage 
processing means can be adq>ted to map image data in polar co-ordinates from the 
sensing means (180^60), or eadi senang means (360?G60) onto adjacent portions of 

10 a Cartel co-ordmate system, so that the (Usplayed unages of the one or two 
hemispherical scenes are continuous. The image data from two back to back imits can 
be displayed in upper and lower portions respectively of a display with a suitable 
rectangular aspect ratio. Alternatively, images from different parts of the panoramic or 
omnidirectional scene (such the reflections in a plurality of facets in a pyramidal 

15 structure) can be routed through respective channels of a video multiplexing device, 
which mables indqp^ent cropping and scaling of the individual imag^ so that each 
can be tailored to fit into the correct locations of the composite image and so that 
adjacent parts of the scene are seamed correctly. The processing means can also 
include means for int^olating the image data to create effects of ^ilargement, 

20 reduction, zoom, pan, tilt, and duinges m aspect ratio. Preferably, the video outputs 
from two or more sensing means which capture respective hemispherical images are 
mapped simultaneously to form a composite itm^e, because this avoids delay in storing 
sequences of video signals (e.g. representing pan, lilt and rotational segments of a scene) 
which need to be separately processed and subsequently assembled. Moreover, in an 

25 omnidirectional optical system, everything is static because no movement is required to 
produce the "all around" 360 x 360 image. In contrast, some prior art systems use 
cameras which are moved or rotated to capture omnidirectional unage information. 

Although particularly suitable for imaging using visible light, the apparatus may be 
30 adapted to image using infra red, ultraviolet or any other fi>rm of electromagnetic 
radiation. 
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In the latter method, opeiBtion can be selected in either a multicast mode^ where the 
whole of the omnidirectional image data is processed and is available for viewing, or a 
sdected zone mode^ where only dififerent parts of the scene are viewed at difiEerent times 
5 and the procesang will then track the movement around the scene. 

In any of the embodiment, shielding means can be mchided for selective shidding the 
imaging unit(s) from one or more predetermined wavelengths of dectromagnetic 
radiation. Preferably, the processing means is adapted to control said shidding means 
10 according to the output ofthe image sensing means. Preferably, the shidding means 
comprises means arranged before, or extending over the exterior surfeces ofthe ima^g 
umt(s) and responsive to input ^gnals for causing local changes in shade or colour. The 
means preferably comprises TOLEDS, OLEDs, FOLEDs or TLCDS controlled by the 
ii^ut signals to provide local changes in shade or colour, defocus, or blur. 

15 

The present invention also provides image projection apparatus conq)iising at least two 
hnage projecting units, each unit including optical means for receiving a respective 
image from image projecting means and ou^utting the image in wide angle form, the 
unages oulput from the respective optical means b dng directed adjacently so as to form 
20 a single composite image when displayed on a substantially spherical screea 

Each unit may comprise a lens system for refracting an image recdved from the image 
processing means on to the screen. The image projecting units may be adjacently 
portioned so as to at least partially encompass the image projecting means. The image 
25 projecting xmits may be arranged back-to-back, with a common axis of symmetry, so 
that each unit outputs an image of a respective part of a scene. The image projecting 
means may be arranged to output images substantially perpendicular to the axis of 
symmetry of the image projecting units, the apparatus comprising means for reflecting 
the unages output from the image projecting means on to the optical means. 

30 

Another aspect of the mvention provides omnidirectional imagSng s^paratus for viewing 
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a panoramic scene and for displaying an image of tlxe scene in an image plane; the 
apparatus comprising optical means for receiving radiation from the scene and for 
directing it onto a focal plane; sensing means responsive to radiation incident on the 
focal plane to gen^ate imagiog signals; processing means for providing data in ^diich 

5 values of the imaging signals, for radiation fiom diflferent elements of the panoranaic 
scene, are related to corresponding positions of incidence of the radiation on the optical 
means; the processing means also producing drive signals ^faen the values of the 
imag in g signals differ from a threshold value; and shidding means arranged before the 
optical means for selectively passing or blocking some or all of the incident radiation; 

10 the shielding means bdng responsive to the drive signals so as to pass or to block the 
incident radiation for wliich the vahies of the respective imaging signals differ from the 
threshold. 

In the latter device, the shielding means can inchide a screen or array of LEDs, such as 

15 TOLEDS,OLEDs^ FOLEDs or TLCDs, whidi can be selectively activated, for exanqple, 
by energising ortiiogonal electrodes in an dectrode miatrix, in order to activate sdected 
zones oftfae screen or array, eith^ to pass^ or to blodi:in(^GQtradiati^^ SudiasGieati 
or array may, for emnple, be normal^ transparent to radiation, xvfaereby sdected areas 
can be activated, by drive signals applied to the dectrodes m ord^ to cause local 

20 changes in optical properties, such as shade or colour, which affect the opadty of the 
screen or array, or the focus, at that locatiort An independent sensor and or the 
processing means can be made to detect radiation by responding to imagmg signals 
having values greatly exceeding a normal working threshold, so as to rapidly shutdown 
part of the screen or array to prevent such radiation from reaching and thereby 

25 dama^ng the sensing means. This could be the case where, for example, a beam of 
laser light is directed to the apparatus to cause damage and the shielding means blocks 
light of this frequency, where it is incident from the panoramic scene. This would 
prevent (say) a CCD sensor from bdng damaged by over exposure. The change in the 
optical properties could, in such a case, introduce a filtering d9fect to cut out light of the 

30 laser wavel^igth. Alt^natively, in a case where radiation is more generally inddent on 
the apparatus, which could cause damage, the processing means can activate the 
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shielding means so as to shutdown most of the inddent radiation, apart from radiation incident 
on a selected part of the apparatus, for example, which is coming from a minor part of the 
panoramic scene for which continued observationis necessary, whilst blocking out background 
radiatioa Clearly, the activation of the screen or array will depend on the application and 
radiation can dther be blocked or passed as may be required over a small inddent area, or a 
large one, or different areas may be activated or not. 

Reference is also made to US 4848890(Hom) which idiows a fonn of sensor control. Also, 
in »nbo(Uments of the invention, the shidding means can be controlled in the manner of an 
automatic iris, e.g. by switching on and of^ sdected zones of the screen, so that thqr fonn a 
pattern correq)onding to the size of the iris which would be adjusted to compensate for the 
brightness of the scene^ and/or flare^ etc. The iris size could be controlled by an external 
sensor, or by TTL (through the lens) metering, as known in the art, or by using the digital 
signals to generate something, such as a histogram, from which brightness levels can be 
calculate. A TTL sensor can be built into a lens system, or in a mirror or prism, in 
embodiments where these optical components are in the light path to the image sensing means. 

Genially, in the prior art the task of capturing panospherical or omnidirectional images is 
often performed by specially designed fisheye lenses, but these have the disadvantages of 
lengthy backworkings and expense. Various aspects of the invention, described above, offer 
solutions to this probl^ by using convex reflectors, some of which have planar reflecting 
&cets. Whilst these provide a more cost-effective solution and avoid spherical distortions 
thereby redudng the amount of processing required with fisheye lenses, there is still the 
problem of using a dngle imaging unit in the att^npt to capture as much of the hemisphere as 
possible (or sphere with back-to-back units). For example, with fisheye lenses, the spherical 
distortion gets worse the nearer the circumferential edge is approached. Hence, the quality of 
the image will degrade the more that it is derived from the edge portions. Moreover, there will 
need to be some compromise in designing the lens so that there is not too much distortion in 
the more central regions of Ihe lens. It would therefore be desirable if the lens could be designed 
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to cope more with a smaller field of view, as long as the rest of the panorama could be 
imaged. 

Anotha* aspect of the invention seeks to solve this problem by providing omnidirectional 
5 imagii^appanituscompristi^ first optical means hav^ second 
optical means having a field of view extending over a segment of a hemisphere, and 
means for optically combining light received from both fields of view for the purpose 
of g^erating an image signal output, the first and second optical means being co-axiaQy 
disposed so that their combined fields of view are hemispherical. 

10 

By suitably arranging tiie optical geometry of the first and second optical means, so that 
th^ each capture optimum images in the respective annular and segmental fields of view . 
and by opticalfy combining this image information, a composite image of the scene can 
be provided in which optical distortions are avoided or substantially eliminated. 

15 

In dthff 180 X 360 deg, or 360 x 360 deg viewing, the (or each) hemisphecical field of 
view is effectively split into two segments, or sub-fields, the annular sphedcal field and 
the segmental spherical field, which are parts of the same hemispherical field of view. 
An important advantage of this arrangement is that the optical means used to gather 

20 images in one or both sub-fields can be of the kind that introduces littie or no optical 
distortioiL By contrast, in the prior art, back-to-back fisheye lens configurations 
produce spherical and barrel distortion, plus chromatic aberrations, but in a flat image 
plane which displays a panoramic view as a composite montage, the central image 
section is the most important for survdllance, but is often the most distorted.. Fisheye 

25 lenses also have severe problems in the secondary spectrum of lateral colour, causing 
chromatic variation or distortion, due to extremely wide angles of the optical system. 
On the other hand, the invration seeks to provide image gathering, espedally in 360 x 
360 deg., whilst preserving integrity of at least the central section and thereby allowing 
limited distortion only in peripheral or top or bottom areas. However, in some cases, 

30 distortions can be reduced to a negligible value. 
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In particular, in one embodiment, an ordinary high quality lens can be used to capture 
images over the central hemispherical segment, thus avoiding the use of fisheye lenses 
(which image the whole h^nisphere). Alternatively, where veiy high quality imaging 
is required, extreme^ high quality specular reflective surfaces can be used (m place of 
S a ceaitral lens), induding parabolic and hyperbolic miiror combinations (similar to 
Cassegrain objectives) to avoid the distortions caused by glass lenses. Hence, the 
invention can be embodied to avoid or reduce spherical or barrel distortion and to 
substantially reduce spherical 1^ aberrations, and distortions and chromatic aberrations 
(e^erienced with fish^e lenses or distorted viewpoints from multiple lens 

10 configurations). This is particularly advantageous with high resolution surveillance 
applications employing high resolution CCDs available from manu&cturers such as 
Phillips and Loran FaircMd. A fiirth^ advantage of some embodiments is that they use 
reflectors which act also as shields to prevmt 1^ flare. This would be the case with, 
for example, apparatus used in daylight surveillance when the sun can cause flare due 

IS to a fisheye 1^ which is directed at the sky segment. In some embodiments of the 
invention, an uppar planar, concave or convex reflector acts as a shield to prevent flare. 

In one embodiment, the annular field (provided by the first optical means), bidudes a 
convene reflector which reflects light firom the scene onto a planar (or concave) reflecting 

20 sur&ce, which in turn directs light through an aperture in the convex reflector. This 
field of view can capture unages from (say) the sides of a panoramic scene where the 
omnidirectional apparatus is suq>ended from a high point in a survdllance zone. The 
segmental field (provided by the second optical means) can inchide an ordinary wide 
angle or other lens (thereby avoiding fisheye lens distortion) located in an aperture in 

25 the planar (or concave) reflecting surface. This captures images from a segment of the 
h^nisphere above (or below) the scene. This lens also directs inddent light through the 
same aperture in the apex of the convex reflector. Thus, light pasmg through this 
aperture, incident on both the first and second optical means, can be recdved by an 
imaging surfiu^e, either directly, or via an optical system which reflects or refracts or 

30 conducts the combined indd^ light from both fidds of view to a more convenient 
location (at which an imaging device is located). 
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According to another embodiment, the hemispherical segmental field of view (in the 
second optical means) is provided by high quaUty concave and convex reflecting 
surfaces arranged, for example, as a parabolic/hyp^bolic primary/secondary mirror 
5 combination to remove optical distortions. This niircor combination can replace a lens 
in the previous embodiment, or can inchide a more simple lens of high qualiQr for 
improving the focusing. This segmental field of view can also be captured by a 
convex/convex reflecting surfiice combination, the sur&ces bdng parabolic and having 
a curvature to provide suitable imaging. 

10 

According to a fiirther embodiment, the first optical means comprises a first convex 
reflector for reflecting Ught through a planar (or concave) reflecting surface, which in 
turn reflects light through an aperture or apertures in the convex reflector to provide the 
annular field of view (as in the latter embodiments). However, the segmental field of 
15 view (provided by the second optical means) employs a second reflecting sur&ce (which 
can be planar or concave or convex, e.g. parabolic) which reflects light onto a second 
convex reflector, which in turn reflects light through the aperture(s) in the first convex 
reflector. This device is of more dmple construction and may not offer as much 
reduction m optical distortion, but it is usefiil in redudng or eliminating lens flare. 

20 

Another embodiment uses an imaging device which fiices (downwardly) a convex 
reflector to capture images in the annular segment, and another imaging device which 
faces (upwardly) to capture images in the upper segment; these imaging devices can be 
inexpensive back to back CCD cameiBs. 

25 

The convex reflectors used in providing the annular field of view can be any of those 
described above in connection with the back-to-back convex arrangements including 
those having first and second reflecting means, the first being convex for reflecting an 
image firom a panoramic scene onto the second which, in turn reflects the hnage onto 
30 the imaging sensing means. For example, the first reflecting means can be 
hemispherical, parabolic, hyperbolic, ellipsoidal, or of a polygonal type where the 
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polygon inchides a plurality of planar or curved refLecting facets surrounding a central 
axis through the convex reflecting means. These can be whole or frusto-convex 
(tnincated) sections. The second reflector can be planar, concave or convex and any of 
the hemisphfidcal, parabolic, i^eEfoolic, ellipsoidal shapes. Where the first and second 
5 reflecting nieans have a curvature, ttiese are in confocalrelat^^ Convex reflectors 
having inclined flat sides in a polyhedral structure, sudi as the six to eight ^ded 
pyramids, for 180 x 360 viemng , or eadi back to bads: for 360 x 360 viewing are 
advantageous (as plained above) and because curved reflectors offer lower optical 
quality compared to plane minors. For example, with a parabolic convex reflector used 

10 in surveillance^ the equatorial images are more in^)ortant (because they view a zone on 
the land) than those at the poles (which view the sky). However, these equatorial 
regions are less well imaged in the sides of the parabolic reflector due to spherical 
distortioa This aspect of the invention enables the image quality in the equatorial 
regions to be improved, e.g. because planar &cets can be used to reflect the side images 

IS onto the imaging means and a simple low quality lens can be used to image the polar 
regions. 

By suitably arranging the optical geometry of the first and second optical means, which 
can inchide selecting the size and curvature of reflecting sur&ces, the size, power and 

20 prop^es of lenses, the relative dispositions of the optical components including 
spadng therebetween and aperture width, images fix>m both the annular and spherical 
segments of the h^nispherical field can be captured and optically combined to provide 
a composite optical image of the scene. This geometiy can be designed, or made 
adjustable, as explained above in connection with other aspects of the invention, 

25 inchiding suitable mechanical drives for adjusting the relative positions of lenses, 
reflecting surfaces, optical co-axial alignment, both to provide a common boundary 
between the image from the annular field and the image from the segmented spherical 
fidd, as weU as conipose the respective images on the viewing plane. Thisseamsmay 
be made inviable, or they can be viable. Instead of making adjustments of the image 

30 optically on the imaging sur&ce, image procesang techniques can alternatively be used 
which can recognise a common boundary in providing a composite image^ dther with 
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or without seams. Suitable available software and programming techniques for 
achieving the required compo^ image are well known to those skilled in the art. 

The imaging units described above can be used with means for atfachitig them to a 
5 camera to provide 18(toc360 or 360x360 imaging, e.g. wherein convex reflectors are 
arranged back to back and a laterally extending member projecting through the convK 
reflectors provides an optical and mechanical camera attachment. 

Embodiments of the invention will now be described with reference to the 
10 accompanying drawings, in whidi:- 

Figwre 1 is a schematic drawing of an embodiment of an imaging apparatus; 

Figure 2 illustrates the upper and lower hemispherical fields of view of back to back 
15 convcK reflectors (or fisheye leases) aad how these are developed into a cylindrical field 
of view which is then xmroUed into a flat plane; 

Figures 3a and 3b illustrate respectively the view fi-om each convex reflector (m a flat 
display plane) and the corresponding developed upper and lower fidds of view in a flat 
20 plane after mapping the image signal data firom polar to cartesian co-ordinates; 

Figure 4 is a mapping diagram for the koages of Figs 3a and 3b; 

Figure 5 is a schematic drawing of another embodiment of imaging apparatus; 

25 

Figure 6 is a sch^natic drawing of a stareoscopic arrang^mt of the imaging ^aratus; 

Figures 7 and 8 are sectional and perspective views of imaging units whidi employ 
convex pyramidal reflectors (suitable for pipe mspection); 

30 

Figure 9 is a section of a stereoscopic version of Fig. 7; 
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Figures 10 is a perspective view of anollier pytamidal convex reflector atrangemeiitwith 
more sides; 

S Figures 11 and 12 are sdheoaadc diagrams illustrating mapping for a back-to-badk 
pyramidal structure with dght sides; 

Figure. 13 illustrates a back4o-badcfisheye lens arrangement 1^ 
sensing sur&ces; 

10 

Figure 14 is a perspective view of an embodiment in which a motion sensor provides 
position related signals for navigating around a panoramic scene on a display screen; 

Figures 15 and 1 6 are perspective and sectional views of another pyramidal arrangem^ 
IS with upper and lower reflectiijg systems; 

Figure 17 and 18 show a modiflcation with curved sides; 

Figure 1 9 shows a fijrther pyramidal embodiment, and Figure 20 shows a back-to-back 
20 arrangement of the same embodim^ 

Kgure 21 shows a further modification of the embodimmt of Figure 19, whidi includes 
a wide angle laos for viewing an upper segment of a hemisphere, where the pyramidal 
reflector views an annular segment of the hemisphere; 

25 

Figure 22 shows a back-to-back arrangement of Figure 21; 

Figures 23a-23c are mapping diagrams for the embodiment of Figure 22; 



30 



Figure 24 shows a similar embodim^ but using optical fibres, Figure 25 illustrating a 
manifold; 
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Figure 26 shows a modification using optic fibres; 

Figure 27 shows back-to-back fisheye lenses with CCDs in between; 

5 

Figure 28 shows a modification vAxsrt each fisheye lens is effectivdy separated into two 
parts to reduce the length of backworkings; 

Figure 29 shows a variation with a prism fot a reflector, 

10 

Figure 30 shows the arrangement of Figure 28 in a housing, and Figure 30a shows a 
similar arrangement for ISOKSdO"" viewing, e.g. for melting on a cdlmg; 

Figures 3 1 and 32 schematically illustrate an arrangement for dealing with lens flare; 

15 

Figures 33-35 show other back-to-back fisheye lens arrangements; 

Figures 36a-36b is another mappiiig diagram; 

20 Figure 37 shows a projector arrang^ent; 

Figures 38-42 show fiirther embodiments of anangmients v^ an upper segmental field 
of view and a lower annular field of view (in each hejmisphere); and 

25 Figures 43 and 44 show 360x360° and 180x360*" arrangements with iimer planar 
pyramidal reflectors and outer parabolic convex reflectors. 

With reference to Figure 1 one fflibodiment of an imaging apparatus 10 comprises two 
imaging units 12, 12*. Each imaging unit 12 comprises an image sensor 14, such as a 
30 CCD or camera, a convex parabolic reflector 16 and a circular, substantially planar 
reflector 18. The apparatus is housed in a transparent housing 19, but the transparent 



wo 01/68540 PCT/GBOl/01115 

-23- 

housing is optional and can be dispensed with. Alternatively, a solid optic can be 
employed, e.g, which is made of transparent plastics ext^ding between the surfaces of 
the parabolic and planar reflectors (16,18). The reflecting surfaces can be formed on 
polished end sui&ces of such a plastics body. The housing (or outer surface of a sohd 
5 optic) is pref&rably symmetrical with the shape of the convex reflector so as to avoid 
optical distortion of the ligjbt incident from the scene. 

The camera 14 and the planar reflector 18 are both positioned along the axis of 
symmetry 20 of the parabolic reflector 16, the surface of the planar reflector being 
10 substantially orthogonal to the axis of symmetiy 20. As shown in Figure 1^ the CCD 
array or camera 14 is situated within the parabolic profile of the parabolic reflector 16, 
so as to provide a relatively compact structure for each imaging unit and so that the 
parabohc reflector 16 can provide a relatively secure housing for the camera 14» Use 
can also be made of telecentric optics for reducing perspective distortion 

15 

The convex parabolic reflector 16 is positioned to reflect an image of a sc^e on to the 
planar reflector 18. The planar reflector 18 in turn reflects the reflected image througjbi 
an aperture 22 formed in the paraboUc reflector 16 to the CCD or camera 14. Alens 
24 is provided between the CCD or camera 14 and planar reflector 1 8, for example^ in 
20 the aperture 22 formed in the parabolic reflector 16, to focus the reflected image 
reflected by the planar reflector 18 on to the CCD or cam^ 14. The lens may be 
provided with any suitable means for blocking ligjit which has not been reflected by die 
planar reflector. 

25 The planar reflector 18 is mounted to the lens 24 of the camera by a shaft 26 extending 
along the axis of symmetry 20. A motorised gear system, such as a rack and pinion 
arrangement (not shown), may be provided to move the shaft 26 along the axis 20 in 
order to vary the optical distance between the parabolic reflector 16 and the planar 
reflector 18. For exan^le, decreasing the distance between the two reflectors will have 

30 the effect of redudng the size of the image reflected by the planar reflector 18, and 
concomitantiy reducing the size of the scene imaged by the imaging unit, thus enabling 
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a user of the apparatus to "focus in" on a particular feature of the imaged seme. The 
planar reflector can be replaced by a concave reflector in confocal relationship with the 
convex reflector to reduce optical distortioa The reflecting sur&ces can be movable 
or adjustable^ as mmtioned above> to assist in composing te final imagp on the viewing 
S plane. 

In the apparatus shown in Figure 1, the two units 12 are arranged badc-to-back, with 
the two parabolic reflectors 16, 16' sharing a common axis of symmetry 20. The 
parabolic reflectors 16, 16' may be joined, as indicated at 28, to provide a seal to prev^ 
1 0 ix^ess of water or debris, and to prev^ any other form of access; into the housing 3 0 
thus defined hetwe&x the parabolic reflectors 16^ 16*. 

The CCDs or cameras 14, 14' have image signal outputs connected to image procesang 
apparatus (not shown), such as a suitable programmed computer, for processing the 

IS image signals for display on a display device (not shown). These procesang and display 
tedmiques are well known in the art. The image processing apparatus transforms each 
of the outputs firom the ima^ng units into signals into image sdgoal data. Figure 2 
illustrates an example of the respective spherical unage signal data 40, 40' produced 
fix)m the image signals ou^ut fix)m the cam^:as 14, 14' resfpectivel/ wh^ the apparatus 

20 is utilised as part of a survdllance system. Each CCD or camera 14, 14' views 
hemispheres of the scene in respective parabolic reflectors 16,16' and produces analog 

^ ^ ^ ^^^^^ ^^^ ^^ ^^^D^^^^^^ 

which are converted into digital signals representing a substantially 
circular image 40, 40' of each hemisphere 50,50' of a scene. As shown in Figure 3a, a 
portion 42, 42' of each of the circular images 40, 40' is masked by the planar reflector 
25 18, 18', the size of these "blind spots" 42, 42' depending on the size of the planar 
reflectors 18, 18' and the distance between each planar reflector 18, 18' and the 
associated parabolic reflector 16, 16'. 



As will be readily appreciated, it is difflcult for a viewer of a display device displaying 
the circular images 40, 40' to identify eaafy particular features of the scene. Therefore, 
the image processing £q;>paiatus performs a msq;>ping operation on eadx image signal data 
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for transformation of the data into a cartesian co-ordinate system for ou^ut to the 
display device. This mapping operation is iUustrated in Figure 3. 

Coiiq>aring Figure 2a with Figure 4, in the mapping operation, each of the circular 
images is notionally divided into an array of pixds (the gcid overlaying each of the 
drcular images 40, 40* should be ignored for the present purposes) in a polar co- 
ordinate systeno. Each ofthesefdsds is then nipped, using look^ip tables stored m 
image processing apparatus which also compensate for distortion resulting from the 
nmpping, into a cartesian co-ordinate system for dispky in a rectal^ In 
this mapping tedinique as shown, in Figure 4, four circular sectors, each numb^ed 1-4 
and 5-8 in the upper and lower hemispheres of view, map to rectangular areas 1-4 and 
5-8. The technique is particularly suitable for mapping of the circular images 40, 40* 
into adjacent portions, such as respective upp^ and lower portions, of a 16:9 
rectangular video display. Figures 3a and 3b illustrate tfaree-dimensionally the result of 
the mapping operation, in which two substantially hemispherical, or "omni-spherical**, 
images SO, 50* are transformed into a continuous cylindrical "pano-spherical" image 60. 
This is unrolled into a flat plane as shown in Figure 2b and is the resultant 
transformation of the displayed circular images 40, 40', providing a continuous two- 
dimensional image of the two scenes imaged by the imaging units. The resultant 
rectangular image 70 enables theuser of the apparatus to more readily identify particular 
features of the scene. This enables the us^ to magnify selected features of the di^layed 
scene and to pan the scene in any desired manner by suitable digital processing. The 
image 70 may also be transmitted as a flat panoramic image and interpreted by a d&ssA, 
or remote computer software or driver, as a navigable scene which may be cropped, 
enlarged or rotated in real time in response to control inputs from a remote user. 

As the user zooms in on smaller portions of the scene, the granularity of the viewed 
image will increase. Tfaus^ the apparatus may be programmed to control movement of 
the planar reflectors relative to the parabolic reflectors with the magnification of the 
image by the user to compensate for this loss of resolution 
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As shown in Figure 5, the planar reflectors 18, 18' may be attached to the inner surface 
of the transparent houiring 19, thereby avoidhig attachment of the planar reflectors to 
the parabolic reflectors via shafts 26. 

The reflectors 18, 18* may be shaped and arranged in confocal relationship (e.g. 
concave^convex) to assist in redudng optical distortion. Eadi imag^ seising means 14, 
14' may comprise a CCD airay with a convex or concave image recdndng surfiu^e to 
reduce optical distortion. A polarising filter may be located between each cam^ 14 and 
reflector 1 8 to reduce the incidence of reflections fi-om the housing 19 on the cameras. 

Instead of using single lenses 24,24' and respective cameras 14,14', these can be 
replaced by double lenses 24a,24b and 24c,24d and respective CCD devices 14a, 14b 
and 14c, 14d, separated by light shields 25,25', as shown in Fig 6, for stereoscopic (or 
stereographic) omnidirectional imaging (see below). 

Figure S also illustrates that hou^ng 19, having a wall thickness 19a, can be replaced 
by a soHd transparent optic 19b, which ext^s between the boundary 19c and the 
reflecting sur&ces 16 and 18 (e.g. made of dear plastic, or crown optical, or 
germanium.) 

Fig. 6 shows an arrangement similar to that of Fig. S, except that two lenses 24a, 24b 
and 24'a, 24'b, are positioned in the openings at the apex of each triangular arrangement 
16a, 16'a, and that respective imaging devices 14a, 14b and 14'a, M'b, receive the light 
from the respective lens. Such an arrangement can be used, for example, where 
simultaneous infrared and video images are sensed of the same scene^ or where stereo 
video images are produced. The imaging devices can be cameras, but are preferably 
CCD arrays. This arrangement is also suitable for st^eoscopic (or stereographic) 
omnidirectional imaging where the lenses and imaging devices receive light from two 
vantage points for the purpose of creating a stereoscopic inuige, by known means (such 
as tiiose desaibed in ''Advanced Photography, by Michad Langford, ISB 024051029- 
1, or ''Autostareoscopic Computer Displays" made by manufacturers such as Philips or 
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Sharp. Hence^ it becomes possible to view any part of a panoramic scene in 3D. 

Fig.7 shows another embodiment of the invention where eadi imaging unit includes a 
convex pyramidal arrangement with four spherically triangular planar mirrors as the 
ffldesl6a Qiistead of u^ a convex spherically parabolic reflector 16). Sides 16a are, 
in &ct, quadrilaterals and reference will be made, in general, to fitcets of a polygonal 
structure. The most sm^)Med embodiment of a convex pyramidal structure has 4 sides 
and this will described to iUustrate the prindples^ because pyramidal structures with 
more sides are preferred for viewing distant objects (the further away the object, the 
more sides or facets of the pyramid are required, and a preferred number of sides are 
six-eight in each pyramidal structure, i.e. twelve - sixtem in back-to-back 
arrang^ents). The arrangement shown in Figure 7 has a practical use for objects vrfiich 
would be near to the reflecting surfaces. For example, it can be used in pipe inspection, 
where the back-to-back jinagjng units travel down the pipe in the direction of the 
longitudinal axis through the lenses 24,24*. As in the previous embodiment, a 
substantially planar reflector 18 and image sensor 14 are positioned along the axis of 
symmetry 20 of the reflecting arrangemmt, the sur&ce of the planar reflector 18 being 
substantially orthogonal to the axis of syrtmietry 20. Fig. 8 schematically shows two 
such imaging imits 12, in perspective, in a back-to-back (360 x 360) arrangement, the 
lenses being omitted for clarity. The aides or facets 16a form an octahedron, open at 
each end 16b, 16*b, (so as to receive a respective lens or lenses 24,24'). The imaging 
devices 14,14' can be cameras or charge coupled devices, CCDs, which are mounted 
with respect to the lens 24,24^ in the manner described above. The same or a similar 
motorised gear system can also be used to vary the optical distance between each 
pyramidal reflecting arrangement 16a,16'a and its respective planar reflector 18,18*. 

Fig. 9 shows an octahedral planar mirror arrangement for st^eoscopic viewing, with 
two sets of lenses 24a,24b, and hnage smang means 14a,14a', 14b,14b\ This 
arrangement is similar to that of Fig. 6 and it can be used for the same purposes. 
Fig. 10 shows a fiuther embodiment where the four-sided pyramidal planar mirror 
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arrangement (4 facets) is rq)laced by a six-sided pyraniidal planar minor arrangemeaits 
(6 facets) 16d, 1 6d\ The coxtstruction and operatioB is otherwise similar to that shown 
m Figs. 6 and 8. 

The moping operation for the embodiments of Figs. 6-10 are idmilar, in piindple, to 
those already explained above, but as they take account of the octahedral and 
dodecahedral shapes (8 and 12 £icets) instead of the parabolic sur&ces, they can 
significant^ reduce the amount of processing required for transforming (otherwise) 
spherical images (&om parabaloids) into flat images which need to be composed on a 
flat image plane with little or no spherical distortion For example. Fig. 1 1 shows the 
octahedral planar &ces designated as regions 1 to 8 in polar co-ordinates (Figures 12a 
and 12b) which are mapped onto cartesian coordinate flat planes as shown in Fig. 12c. 
As shown in Fig. 12c, the horizontal rows represent a 360° horizontal view, whareas 
the cokmms represent the 360° (+/-) vertical view. The mapping opearalion can be 
carried out with the aid of coordinate lookup tables whereby the resultant 
transformation of the displayed images in the planar ficets reconstruct the required 
composite visual image. Any suitable known technique can be used for processing the 
digital signals so that the unages in the &cets are cropped, scaled and stitched together 
to provide the required panoramic image in the viewing plane (e.g. suitable techniques 
are those knovm as "^ples Quidctime VR" and as disclosed in McCutcheon in US 
6141 034). Whilst planar &cets may not provide the same Add of view as the parabolic 
reflectors, they are of far simpler construction and will provide images of visually 
acceptable quality, over a sufficient field of view of interest, so that much less 
processing is required for image transformation and much less cost Image processing 
techniques can be used to refine the pixel information so that the final composite image 
is according to requirmient. 

CCDs can be used of 4:3 proportion or 16:9 proportion, or a split 16:9 proportion, but 
oth^ combinations can be used. The resultant output or display can then be, for 
example, 4:3, or 16:9 mtio. Howev^, any aspect ratio can be used m accordance with 
us^ requirements. 
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Whilst the above example has been described with a pyramidal structure having four 
sides, more sides are preferred especially for viewing distant objects. For example, 6-8 
sides or ficets in each imaging unit (12-16 in the back-to-back arrangement) provide a 
usefiil field ofview and acceptable inoag^. Qeai]y,astiienumberofficetsisina:eased, 
the pyramidal structure becomeis more circular and this improves the panoramic view 
of the scene. Although the &cets are inclined at say 45^ (and they can be inclined at 
other angles), it is surprising v/bat can b e seen in the reflections from adj acent &cets and 
adjustments can be made so that there is sufficient overlap of images in adjacmt fiu;ets 
to enable parts of the digital image of the scene to be seamed or stitched together to 
provide the continuous panoramic image. The mapping of N fecets onto N regions of 
the composite image will follow the same principles as those e^lained herein. 

The arrangement shown in Figs. 6-9 can be contained in a transparent spherical, 
cylindrical, tubular or triangular enclosure and the enclosure, preferably cylindrical or 
spherical, can be mounted on shafts which are driven by a motor in order to spin Ihe 
enclosure to remove, e.g., rain or dust, or oth^ polhitants, by centrifugal action. 

if necessary, section or portions of the convex mirror arrangemmts, e.g. an area or 
selected areas on the sur&ce of the parabolic convex mirror, or the suifiice(s) of the 
planar mirrors in the sides of the polyhedral convex arrangements can be obscured, 
defaced, removed or otherwise treated to mask that part of the scoie firom which light 
is reflected, hence blocking the view of the imaging smsor (CCD) to prohibited viewing 
zones. This allows for privacy in survdllance applications and can provide a satisfactory 
solution to the problem of controlling restricted or limited viewing of an omnidirectional 
scene, whare the mirrors would normally be inaccessible once mounted for surveillance. 
(Otherwise, in the case of inhibiting or suppressing the proces^ng of parts of the image 
information to block parts of the view, the procesang of all the unage information could 
still be continued illegally, i.e. without authorisation) 

Whilst the imaging information from the sensors 14,14', are processed to provide the 
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composite image, means may also be provided to operate each image sensor 
indeprndendy of the other. Each imaging unit can also have one or more image sensors 
to provide stereo video stereographic/ stereoscopic three-dimensional imaging and also 
to inq>lement dual infirared and standard video inuige gathering and/or to provide for 
high and low resolution CCD combinations. 

The planar reflector 18,18', also need not be planar, since it can have a curvature with 
is useful for a particular qiplicatioa Where the imaging units are contained in a 
transparent endosure, polarising filters can also be used to rmove reflections fix>m the 
transparent material of the enclosure. The planar reflecting &cets can be square, 
circular, triangular or octagonal in shape. 

Fig. 13 shows another embodiment which is based on the same general principle of 
providing an image sensing device within a light collecting arrangement for producing 
a 360** horizontalAreartical composite image. However, this raibodiment employs lenses 
instead of parabolic or triangular planar rdlecting sur&ces and also employs a special 
CCD arrangement. More particularly. Fig. 13 shows an omnidirectional imaging 
apparatus including two imaging imits 12,12', each unit having a wide angle convex: 
"Fisheye" lens 31,31', and a secondary lens system 32, 32' (which can be concave or 
convex) arranged on the same optical axis. The inwardly and oppodtefy feeing concave 
sur&ces of lenses 32,32', substantially surround a CCD array 33, 33' with oppositely 
fadng paraboloid (or otherwise sphaically shaped) imaging sur&ces. This is a purpose- 
made imaging device . The ejcact shape ofthe imaging sensing surfeceofthe CCD will 
depend on the nature of the optical lens or reflecting arrangement which is used to 
capture the image of the scene, smce this shape is intended to avoid or reduce the 
amount of processing of the digital signals which would otherwise be requked for 
making transformations or waiping, in order to correct for spherical distortion. 
Moreover, the use of a curved imaguag sur&ce makes more effid^ use of the pixel 
area, since no pixels are wasted in an arrangement where the optical image capturing 
device directs or focuses an appropriate image on the curved sur&ce. Intfaecase\^di^ 
the CCD array is flat and rectangular, some areas (and therefore their pixels) are 
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redundant and hence wasted. 

The mapping opei:ation and formation of composite images vnR also be general^ similar, 
but this arrangemrat, being spherical, solves problems of mapping which occur with 
planar sur&ces that need to be transfonned into a single image plane. 

Figure 14 shows a user wearing headgear 40 to wfaidi is attached a device 41 (ofknown 
construction) contaimng giros and accelerometers that detect movement of the head and 
generate corresponding position related signals, ref^enced to an artificial horizon, and 
send them to processing means 42. Device 41 could be described as a "3D mouse", 
since a "2D mouse" provides position data in co-ordinates in a flat pl^ and device 
41 adds the third axis z or third degree of freedom. Further details of such a Head 
Mounted Device (HMD) can be seen in US-A-5645077, US-A-5807284, and 5440326, 
the contents of which are include by cross-reference. An omnidirectional imaging unit 
43 is positioned at the scene of int^est (not iUustrated) provides a panospherical image 
of the scene and derives image signals fix>m the optical image on the CCD arrays (not 
shown). The processing meatis 42 is programmed to transfbim these image signals into 
image signal data having three degrees of spatial oiiratation, i.e.in the panospherical 
image, which are stored for later reference. The processing means is also programmed 
to use the position related signals from the device 4 1 to select the stored image signal 
data so as to generate a display of at least a part of the image of the sc^ie, so that pitdi, 
roll and yaw of the headgear and hence the head, is transformed into equivalent 
movements of the displayed image on the viewing device 44. The screen 45 depicts a 
circular disc 46 representing the panoramic image of device 43. It also depicts a 
rectangular region which is an enlarged part of the scene shown in 2 dimensions, on a 
flat plane. This part will roll, pitch and yaw to follow the us^s head movements, h^ce 
aUowing all segments ofthe image to be available for viewing. The seme may be static 
or a moving image (e.g. a video recording). 

The latter HMD device can be used by an observer to view any part of a panoramic 
scene, by simply moving the head. Processing of the digital signals can be carried out 
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by known techniques, for example, where the whole panoramic or omnidirectional 
image is fiirst cq)tured as a frame of information, which is then processed and buflFered 
to provide data relating to any part of the scene. If the user wishes to look at a 
particular part of the scene, the bufifered information can then be processed accordm^. 
This can help in reducing processor tasking^ as well as assisting with bandwidth 
requir^ents, and can also allow for ^ihanced resolution from a particular sector or 
zone under investigatioa Some form of switching device can be included for 
automatically switching from the fiill panoramic view to the view of a limited sector or 
quadrant so that the image data of the part und^ observation is ready for immediate 
display. The apparatus can be constructed to operate in dther a multicast mode, i.e. 
where the whole of the panosphedcal image data is processed to enable many users to 
observe (interactively) diflFerent parts of the scene at the same time, and a single 
observer mode, e.g. where one user will only observe different parts of the scene at 
different times and the processing vnH track the movement around the scene. 

Referring to Figs. IS and 16, these schematically illustrate a lower pyramidal (4 sided) 

structure 50 which is designed to rd&ect light from its outer sides or &cets S0a,S0b, 

onto the inner sides or &cets Sla,51b, of a corresponding (4 sided) upper pyramidal 
structure 51. (While 4 sided pyranudswiU be described to sinq)lify the 
drawings, they are most suitable for imaging close up objects, for erample, the interior 
walls of pipes in a pipe inspection tool, and other structures are described which have 
more sides or fecets and are better suited to imaging distant objects. The pyramidal 
structures are located one above the other and their vertical spacing will dq>end on the 
application. The upper segment of the hemispherical scene will be obscured by the 
upper structures, but this may be of no consequence, say in the case of using say 6-8 
sided pyramidal structures 51 for distant objects, for example, where the sky is not 
imaged in survtillance equipment for say a car park.) Theinneriddes 51a,51b each 
reflect incident light onto an opposite iimer side of a downwardly &dng pyramid 52, 

which in tum directs Ught downwardly onto the sides Ma^Hb , of a CCD, sen)^ng 

structure. (The structures 50 and 51 are slices of a regular pointed pyramid, whereas 
pyramid 52 is regular.) Structures 50 and 51 are id^tical apart from the &ct that the 
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outwardly pointing sides or fiicets 50a,50b... are reflective, whereas the inner 
downwardly facing sides or facets 51a,51b... are reflective. While pyramid 52 differs 
in shape, the length of the triangular sides of 50,51 and 52 are the same and all sides axe 
inclined at 45 degrees to the vertical. Similarly, the ^des of the CCD structure 
14a, 14b.... are of complimentary size and shape as shown in the drawing. However, 
instead of having a CCD structure with sloping ^des the CCD can extend in the 
horizontal plane and be either made up of triangular CCDs, or a angle planar CCD with 
its sui&cesub-Klhdded into triangolarsec^ The reason for using sloping sides for the 
CCD structure is to compensate for any trapezoidal distortion caused by the inclination 
of the reflecting sur&ces, with respect to the part of the panoramic scene which is 
reflected and is also imaged on the corresponding CCD or sect^ This 
significantly reduces the amount of processing required for transforming image signals 
to remove any distortion introduced by the optical system in casting images onto 
sur&ces which are not designed to provide the usual image plane. 

Gmerally, if planar reflecting facets are used to capture images from parts of the 
panoramic scene, these planar surfaces do not themselves introduce any distortion 
(compared with the severe spherical distortion which can be introduced by peripheral 
edge portions of a fish^e lens). However, if the image plane is flat, but at an angle to 
the reflecting surBu^e, this can introduce trapezoidal distortioii. This is not by ai^ means 
as serious a problem as dealing with the sphmcal edge distortion m fisheye lenses. 
Th^efore, even with a CCD extending in the horizontal plane for imaging sectors of the 
panoramic scene reflected from the fecets, the amount of processing required to remove 
just trapezoidal distortion is minimal. 

The design ofthe structure shown in Figs. 15 and 16 is such that only light reflected by 
the &ce or &cets 50ais reflected by the optical systm onto the respective idde or sector 
ofthe CCD structure. Therefore, the side or sector 14a, for example, could only see 
light which is derived &om that part ofthe panoramic scene which is first reflected in 
the facet 50a, the same applying to the other faces and CCDs or parts of CCDs. 
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The drawing has been simplified, for ease of understanding, because better results will 
be obtained by using a convex polygonal structure having more than four sides (like a 
many-dded pyramid), snce this wili have more &cets for reflecting respective parts of 
the panoramic scene. For exanq>le, pyramidal structures SO and 51 can be replaced by 
bands having say eight or twelve face^ each band synmietrically endrding the central 
convex axis through structures 50,51,52, and 14. In this case, in order to turn the 
inddent light througih 180 d^rees, after reflection firom the fiuxt (50a, 50b...X ^ 
pyramid 52 would be replaced with a pointed structure having a corresponduxg number 
of facets to provide the same efifect as that shown with the four sided pyramid 
structures. The greater the number of facets, the more the number of parts of the 
panoramic scene will be reflected and these parts will conform in a more optimimi way 
with the continuous spherical image which can be obtained with, for example, 
hemispherical, parabolic convex reflectors or fisheye lenses. Moreover, a greater 
number of facets facilitates reflecting parts of the panoramic scene with an overlap of 
adjacent sides of the &cets. This overlap assists in aligning the reflected images fi-om 
each of the facets in order to provide continuous seams between the parts of the 
panoramic scene. 

In the section shown in Figs.l7 and 18, the fiat &cets are curved, as seen in the 
perspective view of one &ceL Howev^, the reflecting fiicets are flat, ance the spherical 
distortion introduced by the lower convex fiicet is compsosated by the curvature of the 
upper concave fiicet. 

Fig. 19 shows a modification of the pyramidal type of structure where tiie inclined outer 
reflecting faces or facets 60 of a convex regular pyramid 61 refiect light inddent from 
the panoramic scene onto the upp«r reflector 18 which in turn reflects light downwardly 
through apertures 62 in the inclined faces or &cets. Located beneath eadi aperture is 
a camera or CCD sensing system 63 which captures that part of the panoramic image 
reflected by the respective &cet 60 and part of reflector 18. The aperture 61 can be 
open, but in order to hide the cam^ laos firom view, the aperture may be, for example, 
semi-silvered so that it is partly transparent. It can also be dearly transparent. 
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Fig. 20 shows a back-to-back arrangement based on the embodiments of Fig. 19, i.e. 
with two sbc-sided mirrors. Similar parts are gjiven the same reference numerals. 

Whilst Figs. 19 and 20 illustrate respectively six and tweh^e sided reflective structures, 
the number of sides or &cets can be increased to provide a more continuous panoramic 
view. 

Fig.21 shows an arrangement based on that of Fig. 19, but with the addition of a wide 
angle lens system 120 arranged in an aperture 121 in the centre of tiie upper planar 
reflector 122 (similar to r^ector 18 in Fig. 19). This lens systm 120 provides a view 
of a segment of a honisphore, whereas the reflecting arrangement 60, 122 provides an 
annular field of view. The upper segment is imaged in CCD 123, whereas the lens or 
camera arrangements/CCDs 63 are as shown in Fig. 19, and image the scene in the 
annular field of view. An important advantage is that the lens system 120 can be 
selected so that it has a wide field of view but does not introduce optical distortions or 
abemtions over the field of interest^ because the annular field of view of the reflecting 
system 60, 122, can provide an image of the annular s^ment of the panoramic scene 
with little or no distortion. 

Fig. 22 shows a back-to-back arrangement based on Fig. 21, for 360x360 viewing. 

Figs. 23a-23c are mapping diagrams for the structure shown in Fig. 22. In this case, the 
image in the lenses are 120a, 120b have been divided mto equal sa:tors 1 A-IF and 2A- 
2F respectively, whereas the sides of the pyramidal structure are designated 3-8 and 9- 
14 respectively, in the polar co-ordinate diagrams 23a»23b. These map to the cartesian 
co-ordinates shown in Fig. 23c. 

The lens arrangem^ 120 in the ape?cofthe pyramidal structure can be located m the 
other arrangements described below (since they have been siinplified to show the 
pyramidal reflecting arrangement only, where modifications are described below). 
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Fig. 24 is based on the embodimMt of Fig. 20 but instead of camera 63 located just 
bdow the apertures 62,optical systems 64 are provided Tvhich collect incident light and 
channel it through optical guides 65 to a manifold system 66 situated above a CCD 67. 
This embodimmt is useful for conq)osing images directly onto CCD 67 as a montage 
of respective parts of the scene reflected by the fecets 60. Instead of using a single 
CCD, individual CCDs can be located at the outlets of the optical guides 65. 

Fig. 26 shows a variation on Fig. 21 where the reflective fiicets 60 capture parts of the 
panoramic scene and lens systems 68 receive incident light from the reflecting sur&ces 
60 which is then conducted via optical guides 65 to manifold 66 over CCD 67. 

Fig. 25 shows a detail of the noanifold 66 where some light guides 65a enter vertically 
and others 65b are curved because they are directed upwardly from lower reflective 
surfaces in a back-to-back assembly of convex reflectors. In this case, four optical 
guides montage the image from an eight-sided structure. 

The advantage of using optical guides, such as optic fibres^ is that it is easier to 
construct the imaging unit, because a plurality of images can be conducted from 
respective fricets through a smaller q)erture (62) or 70 (Fig. 20) in the systems 
described. Suitable fibre optic image conduits are available from Edmund Industrial 
Optics and these are manufactured with standard and high resolu^ 
fused glass optical fibre rods with round and polished faces. Images are transmitted 
from one polished face to the other and the rods can be bent xmder heat to the required 
orientation. The same Company provides fibre optic tapers and fece plates, where a 
taper is a coherent fibre optic plate for transmitting rither magnified or reduced image 
from its ixsipat to its ou^ut sur&ce and a fibre optic plate is a coherent fibre optic 
plate that predsely transmits an unage from its input surfiice to its ou^ut surface. 
These can be used, for esanq;)le, to transfer Ugh resolution imag^ m CRT/LCD displays 
and CCD coupling. 
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Fig. 27 shows another anbodiment of omnidirectional imaging apparatus wfaidi employs 
lenses in imaging xmits 12, 12*, each unit having a convex: Fisheye, wide angle or 
panoramic lens part 100, 100' and a secondary (concave or convex) lens part 102, 102' 
arranged on the same optical axis. The outwardly facing Fisheye lem systems 12 and 
12* encompass CCD array 104, compriang a pair of badc-to-back planar CCD devices 
106,106'. The imaging units and CCD array nuQ^ be nu>unted so that the optic^ 
has any deared orientation, for example vertical, horizontal or otherwise. An advantage 
provided over previously described embodiments employing mirrors in the unaging units 
is that the lenses 100 and 102 have no blind spots and provide self focusauog. They can 
also be less prone to vibration than the mirrors, inq)Toving performance. By providmg 
the CCD devices 106, 106' between the imaging units 12, 12', ±e CCD devices do not 
obscure the scenes being imaged by the imaging units. By using two CCD devices 
rather than one to collect the image data, the resulting superior resolution of the 
resultant image allows for cropping and scaling of the image in a navigable or int^active 
video. This is desirable as Fisheye lenses have such a wide field of view that mid-range 
objects ^pear distant in the resultant image. To restore normal view, the user would 
zoom or scale the video image, causmg granularity in the inoage. Superior resolution 
provided by two CCD devices rather than one ameliorates this problem. 

A problem of the arrangement shown in Fig. 27 is that the backworkings of each fisheye 
lens is on the same optical axis, therefore making the arrangement, very long and 
cumbersome. This problem is solved in the embodim^ shown in Fig. 28. In this case, 
each fisheye lens has been effectively separated into two parts 130a, 130b, and 
130c,130d, arranged on optical axes which turn through 90 degrees. The outer parts 
of these lenses 130a,130c, encompass an inner region 131 in which a double sided 
reflector 132 is provided (or a prism for the same purpose). This reflects light, for 
exanq)le, fi^om part 130a to part 130b whidh focuses on CCD 133a to provide the upper 
panospherical image. The same applies to the parts 130c, 130d and the CCD 133b. 

Fig. 29 shows a similar arrangement for 360x180 viewing, but wh^e a prism 134 is 
used. 
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Fig. 30 shows a preferred form of housing 135 for the arrangement shown in Fig. 28. 

Fig. 31 shows an embodiment in which the Fisheye lenses 100, 100* are covered with 
an active cover or coating 108, 108' which acts to substantially prevent "leos flare". 
Lens flare occurs when a lens is pointed towards the sun or other brig^ artificial light 
source^ vMck creates a "flare" over the resultant image which can obscure a significant 
proportion of the image. As will be qppredated, if the imaging apparatus is used for 
external surveillance, dq)ending on the atuation of the apparatus lens flare due to the 
sun may be a common occurrence. Further, flare may be induced ddibes^y, for 
example, by a hand*held laso" device, in order to '"bum out" the CCD array 104. 

Since the apparatus is used to provide an omnidirectional image, use of conventional 
lens shades and flags to prevent lens flare is not suitable. Suitable shields 108, 108* may 
be fomied from TOLED (Transparent Organic Light Emitting Diode, as described in US 
patent no, 5,703,436), FOLED (Flexible Organic Light Emitting Diode) or standard 
TLCD (Transparent liquid Crystal Display) in the form of a cover, dome, coating or 
shield arranged before, or extending over the convex outer sur&ces of the lenses 100, 
100' and connected to the image procesi^ apparatus for procesang the image signals 
output from the CCD anay. 

The shields are response to input signals for causing local changes in shade or colour 
ofthe sMdds so as to selectively shield the irQag^lg units 12, 12V This is shown in more 
detail in the perspective view (Fig.38) of the surface of shield 108 (108*), where an 
element 109 has been activated to block lens flare due to (say) the position of the sum 
with respect to the shield. 

As a flare at the extreme end of the spectrum strikes one, or both, of the lenses 100, 
1 00*, it is transmitted firom the CCD array 1 04 to the image processing apparatus where 
its presence is detected from histogram tables stored in the processing apparatus. The 
location of the flare is determined from an image map stored in the lookmp tables. The 
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processiiig apparatus then forwards pixel cx)-or(linates of the flare to a graphics card 
which transmits an appropriate signal to the coating 108, 108* to produce a "mask" of 
100% RGB (red/green/bhie) in the portion of the coating througji which the flare passes 
to strike the lens 100, thereby effectively blacking out the flare. This also reduces, or 
removes, posable refraction around the lenses. 

In the embodiments sbavm in Fig 31, the CCD array 104 comprises a pair of back to 
back planar CCD devices which, in combination with the two imaging units 12, 12', 
makes the ^paratus is rdatively long. A significant gap between the two Fisheye leases 
100, 100', results in a blind spot around the middle of the imaging field In order to 
reduce the height of the apparatus, in the embodiment sho^ in Fig. 33 a double aded 
planar, or non-planar, reflector 1 10 or prism is disposed between the lenses 102, 102' 
at approxitnately 45"* to the optical axis to reflect the images on to the CCD devices 
106, 106* arranged with surfaces substantially parallel to the optical axis and to the side 
of the apparatus. This enables the imaging units 12, 12* to be brought closer together, 
reducing the height of the apparatus and redudng the size of, or completely removing, 
the blind spot around the middle of the composite image. Ih the preferred airanganent, 
the ima^ng units 12, 12' are as close together as possible. Furthermore, as the CCD 
devices 106, 106' are no longer arranged back to back, air cooling of the CCD devices 
is improved, and as the CCD devices have a conomon focus and shared angle of 
incidence, any difEbrence in the view point of the CCD devices is removed. 

This arrangement enables fiirther imaging lenses 1 12, 1 12' to be hiserted between the 
reflector 110 and the CCD devices 106, 106* to reduce image distortion without finther 
increasing the height of the apparatus. The use of wide angle lenses produces heavy 
optical distortion, requiring dgnificant transforming by the image processuig apparatus. 
Indeed, distortion may be such that clos&-up objects can not be accurately imaged. 
Therefore, in the embodiment shown in Fig. 34, each imaging unit inchides a finther 
imaging lens 114, 1 14' to reduce image distortion. As showm in Figs. 34 and 35, the 
reflector 110 may have either a conv^ concave or otherwise diaped reflecting surfaces 
to resolve optical distortion prior to image reception by the CCD devices 106, 106'. 
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The mapping operation for the back to back fisheye enobodiments are similar, in 
priiidple, to those already plained above in connection with Fig 3a. For example, as 
shown in Figs. 36a-36c, the circidar images produced by each imaging unit are divided 
into r^ons 1 -8 and these are mapped onto cartesian coordinate planes as shown in Fig. 
36a (which receives the upper image), and Fig. 36b (which recmes the low^ image). 

These cartesian coordmate planes are then fiirther m^ped onto a m^e cartesian 
coordinate image plane, shown in Fig. 36c» whm the regions 1-8 are assembled 
adjacent one another so as to reproduce a visually acceptable omnidirectional image. 

As shown in Fig. 36c, the horizontal rows represent a 360 horizontal view, whereas 
the columns represent the 360*" (+/-) vertical view. The mapping operation can be 
carried out with the aid of coordinate lookup tables whereby the resultant 
transformation of the displayed images reconstruct the required composite visual image. 

In all of the above embodiments, the imaging units are used to provide images of 
respective scenes which are recdved by image sensors and output to image processing 
apparatus to form a sin^ con^osite image of the respective inoages. In an alternative 
embodiment shown in Fig. 37, which is based on the embodiment shown in Fig. 33, the 
CCD devices 106, 106* are replaced by projectors 120, 120' in the form of a quartz bulb 
122, 122' and transparent LCD active matrix display, or TOL^, 124 for projecting 
respective halves of an omnidirectional scene onto the imaging units 12, 12' for 
projection on to a spherical display surrounding the viewer(sX thus providing the 
viewer(s) with an omnidirectional viewing experience of live or recorded material. The 
display screen may be a conventional white screen covered with a high gain material. 
The viewer(s) would be situated at the centre of the display, accessed by flush fitting 
doors coated witii display material. Cooling for the projectors may be provided by any 
suitable means, for exiample a &n collar located around the outside of the apparatus and 
propelled by an externally mounted motor. 

Figs. 38-42 show attemative embodiments in which a lens arrangement 150 is located 
inanap^ture 151 in an upper reflector 18 for providing an image of an upper segm^ 
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and a convex reflector 16 co-operates with reflector 18 to provide an annular field of 
view. Similar conaponents have been identified by similar reference n^ 
have not been explained in detail. However, the following desodption identifies different 
components which are graerally found in the upper (and lower) region of each 
embodim^ As these are symmetrical, only the upper unit has been described. 

In Fig. 38, convex reflector 151 reflects light from the scene onto a smaller convex 
reflector 152, which in turn reflects light through the lens system ISO onto CCD 153. 

In Fig. 39, the lens system 154 images the upper (lower) hemisphere directly ^mthout 
the need for any other reflecting arrangements. 

Fig. 40 shows a planar reflector 155 which reflects light fi'om the scene onto the small 
convex reflector 1 52 which in turn reflects light onto lens 1 50. 

In Fig. 41, concave reflector 156 reflects onto convex reflector 151, which in turn 
reflects onto lens 150. 

In Fig. 42, convex reflector 157 reflects li^ onto concave reflector 158 vMch reflects 
light to lens 150. 

Generally, where concave and convex: reflecting surfaces are arranged to co-operate, 
they can be high quality reflecting surfaces including parabolic and hyperbolic, primary 
and secondary mirror combinations which remove optical distortions. Also, the convex 
and concave sur&ces can form a reflecting system similar to that known as a Cassegrain 
objective, uang a parabolic concave piimaiy mirror, with a hyperbolic convex secondary 
mirror as described for example in 'Modem Optical Engineering", Warren J. Smith, 
"The Design of Optical Systems", pages 476-478. Also, any reflecting sur&ces used in 
any of the embodimrats described herein can be specular, inchiding vacuum dq>osited 
materials, to achieve high resolution and reflectivity without the diromatic aberrations 
in glass leases. 
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Fig. 43 shows another emboduoteot with split fields of view for 360x360* viewing. In 
this case, instead of using back to back parabolic reflectors for the inner annular fields 
of ^ew, a back to back planar/pyraroidal reflecting arrangement (18,60) is used instead. 
This is similar to the arrangements shown in Fig. 20 and hence the construction and 
operation are the same. Also, in this case, the upper and lower segments of the scene 
are imaged in respective convex parabolic and planar (or concave) reflecting systems 
(16,18), which are similar to that shown in Fig. 5 (except they are now separated by the 
back to back pyramidal arrangements, otherwise the construction and operation of these 
outer, spaced, parabolic/planar reflecting systems is the same). Fig. 44 shows the same 
kind of arrangement, but for 180x360* viewing. 

The various reflecting systems described herein can be combined in any way to provide 
the annular and segmental fields of view (in each 180x360* imaging unit) and the 
embodiments described with reference to the drawings are only exan^les of the 
invmtion. Any embodiment can have different geometries and optical components can 
be ind^endently or mutually movable and the lens system, or sh^es for dispositions of 
the curved and planar reflectors can be varied, so that the respective part or parts of a 
hemisph^cal or spherical scene is correctly focussed and composed on the image 
plane. The sc^e will vary in accordance with the application and it may include 
distant objects, as in the case of surveillance, or near objects (as in the case of pipe 
inspection), or something in an intermediate field, such as the interior or a room or a 
vdude. Also, where vertical arrangements have been described for viewing upper and 
lower fields, these can be turned on their sides for horizontal viewing of the left and 
right hand sides of a scene. Therefore, any changes, modifications, or adjustments that 
are necessary in order to compose the seme correctly will be understood by those 
skilled in the art. Various features are described above in connection with preferred 
embodiments of the invention and those skilled in the art will know that they can be 
used in various other combinations than those shown and described. This also applies 
to features illustrated in the drawings and included in the claims. Therefore, all such 
features can provide basis for other claimed combinations in this application, or any 
divisions. 
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CLAIMS: 

1. Imaging apparatus compri^ng at least two imaging units, each unit induding 
optical means vnih a ^de fidd of view, the apparatus also induding image sensmg 
means for receiving an image from the optical means so as to provide a corresponding 
ou^ut; the imaging units being adjacoati^ positioned (a) so as to encompass dther the 
image sensing means, or image diverting means, whidi div^ images to the image 
sensing means, and (b) so that tiie oiitput can be used to form a single composite image 
in the same plane. 

2. Apparatus according to Claim I, wherein each imaging unit tQcludes first and 
second reflecting means, the first reflecting means being convex for reflecting an image 
firom a panoramic sc^e onto the second reflecting means, the second reflecting means 
being arranged to reflect the image onto the image sensing means. 

3 . Apparatus according to Qaim 2, wherein the first or convex reflecting means is 
hemi-spherical, parabolic, hyperbolic, ellipsoidal, or of a polygonal type where the 
polygon indudes a phirality of planar or curved &cets surrounding a central axis 
through the convex refiectmg means. 

4. Apparatus according to either Claim 2 or 3, wherein die second r^ecting means 
is planar, concave or convex (hemispherical, paraboHc, hyperbolic, ellipsoidal). 

5. Apparatus according to Clahn 4, wherein the first and second reflecting means 
have curved surfaces which are in confocal rdationship or are catadioptric, or form 
Cassegrain objectives. 

6. Apparatus according to any preceding claitn, including a tdecentric lens for 
reducing distortions in light mddent on the inmge sensing means. 
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7. Apparatus according to any of Claims 2-6, wherem the &st convex reflecting 
means has one or more apertures or light ports for receiving light reflected firom the 
second reflecting means, whereby inddent light passes to image sensing means. 

8. Apparatus according to Claim 7, wherein the first or convex reflecting means is 
of the pofygonai type, inchiding a pkirality of &cets surrounding a c^xtral axis through 
the coxsvex, reflecting means;, and wherdn either a central c^erture is provided in the 
apex of the convex reflector, or apertures are provided at the midpoint of each facet. 

9. Apparatus according to C3aim 7, wherein a system for gathering li^t is located 
beneath the or each aperture, and the system includes a light guide, such as an optic 
fibre, for conducting light to the sensing means. 

10. Apparatus according to Claim 9, wh^ein a light gathering system is provided 
baieath each aperture in eadi fiicet and a plurality of light guides are connected to a 
manifold for conxposing respective parts of a panoramic scene (Imaged in the re^ective 
&cets) directly onto the image plane of the image senang means. 

11. Apparatus according to Claim 7, wherdn a light gathering system is located 
directed opposite each respective &cet in the convex reflector, the light bemg 
conducted, via a Ught guide, to the sensing means. 

12. Apparatus according to Claim 7, wherein the convex reflector comprises a first 
set of &cets, which are the sides of the polygonal convex reflector having an axis of 
symmetry, the second reflecting means including a second set of fecets for reflecting 
light through a light port at the apex of the convex reflector, the second set of facets 
being arranged to reflect light which is inddent on them only fi-pm the respective fecets 
in the first set, whereby the image sensing means separately and respectivdy receives 
light fi-om those parts of the panoramic some reflected in the first set of fecets of the 
convex reflector. 
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13. Apparatus according to any of Claims 2-12, wherein the first and second 
reflecting means are mutually adjustable to liable the panoramic scene to be composed 
correctly on the image sensing means. 

14. Apparatus according to CSaim 13, inctading means for manually or automaticatty 
adjusting tiie distance betweoi back to badk imagjsg umts, tbe distance between the first 
and second reflectors in each imaging units, for independently or simultaneously 
pivoting either the first or second reflector about an axis, or tilting the same, or 
displadng the same with respect to one another. 

1 5 . Apparatus accordmg to any preceding daim, wherein the image sensing means 
is pro\dded with zoom, and or is mounted in gimbals for making image adjustments. 

16. Apparatus according to Claim 1, wherein parts of the panoramic scene are 
imag^ by planar facets in a reflectmg system, the reflecting system h&ng adjustable so 
that each adjacent &cet sees respective adjacent sectors of the panoramic scene, 
whereby parts of the scene can be joined together in the composite image. 

17. Apparatus according to Claim 1, wh^ein each imaging unit includes a wide 
angle lens syst^ the lens system encon[q>assing the image sensing means which is 
provided with a spherically curved image receiving sur&ce that is shaped to reduce 
spherical distortions introduced by the wide angle lens system. 

18. Apparatus according to Claim 1, wherein each imaging unit includes a wide 
angle lens system, which encompasses the image diverting means, each wide angle lens 
system being separated into two parts, a first part being located so as to gather light 
fi*om the panoramic scene whereby it is inddent on the unage deflecting means, and the 
other parts being located so as to receive light firom the image deflecting means, 
whereby the length of the back woddngs in the wide an^e lens system is reduced on the 
longitudinal axis through the first parts of each lens system and the image deflecting 
means. 
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19. Imflging apparatus for producing stereoscopic images of a panoramic scene, the 
apparatus induding a reflecting system for receiving light from said scene; first and 
second 1ms syst^ns located in the same plane, the lens systems having respective optical 
axes iwinch are spaced apart in said plane and symmetiicdiMllh an axis throu^ a cen^ 
axis of said reflecting systan, and image smsing means for recdving focussed 
stereoscopic im^es of said scene from said lens syst^ns; said stereoscopic images being 
formed by light with difB^nt optical properties, such as difiTerent colours or 
polarisations, vAi&fSby stereoscopic hnages deiived from the image sen^ng means can 
be supmn^osed to give a 3D effect, whereby all or any part of the panoramic seme can 
be viewed in 3D. 

20. ^paratus according to Claim 19, wherein the reflecting system includes a 
convex reflector. 

21. Apparatus according to Claim 20, fiirther induding a second reflector, planar or 
curved, for reflecting light from the convex reflector through a light transmitting port 
in the apex of the convex reflector. 

22. Apparatus according to any preceding claim, comprising shidding means for 
sdectively shielding the imaging units from one or more predet^mined wavdengths of 
dectromagnetic radiation. 

23. Apparatus according to Claim 22, including means adapted to control said 
shielding means accordmg to the output of the image sensmg means and which acts to 
cause a local change m the studding means, or an overall change^ or a change in selected 
regions in order to act as an iris, or any combination of the same. 

24. Apparatus according to any of Claims 22 to 23 , wherein said shidding means 
comprises means arran^ before, or extending over the exterior surfaces of theimagipg 
units and responsive to input signals for causing locd dianges in shad^ colour, or focus. 
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25. Apparatus according to Claim 24, wherein the shielding means comprises 
TOLEDS, OLEDs, FOLEDs or TLCDS cofltrolled by the mput signals to provide local 
changes in shade or colour. 

26. Omnidirectional imaging apparatus for viewing a panoramic sc^ie and for 
di^laying an image of the scene in an image plane; the device comprismg: 

optical means for recdmng radiation from the scene and for directing it onto a 
focal plane; 

sensing means responsive to radiation incidmt on the focal plane to generate 

imaging signals; 

processing means for providing data in which values of the imaging signals, for 
radiation from different dements of the panoramic scene, are related to corresponding 
positions of incidence of the radiation on the optical means; .the processing means also 
producing drive sdgnals when the vahies of the imaging signals differ from a threshold 
value; and 

shielding means arranged before the optical means for selectively passing or 
blocking some or all of the incident radiation; the shielding means being responsive to 
the drive signals so as to pass or to block the inddent radiation for wUch the values of 
the respective imaging signals differ from the threshold. 

27. Apparatus according to Claim 26, wherdn the shielding means comprises screen 
means, an electrode structure for activating selected elements of the screen means so 
that said elements either pass or block the incident radiation. 

28. Apparatus according to Claim 27, wherdn the screen means comprises 
TOLEDS, OLEDs, FOLEDs or TLCDS and wherdn the dectrode structure is 
re^onsive to the drive signals to provide local dianges in the opticd properties of the 
screen means m order to pass or to block the mcident radiatioa 



29. Apparatus according to any of Claims 26-28, \^erdn the processing means: 
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(a) detects imaging signals having values vAich differ fix)m the threshold by 
an extent that could damage the sensing means; and 

(b) provides the drive signals necessary to activate automatically a minor 
part of the shielding means for blocking the damaging radiation to prevent it bdng 
incident on optical means; ^^4iilst 

(c) enabling non-<!amagjng radiation to pass through tiie shidding means to 
be incident of the sensing means to maintain panoramic scene viewing. 

30. Apparatus according to any of Claims 27-29, wherein the processiog means: 

(a) d^ects imaging signals having vahies whidh difiEer fiom the threshold by 
an extesat that could damage the sensing means; and 

(b) provides the drive signals necessary to activate automatically a major 
part of the shielding means for bloddng the incident radiation to prevent 
it being recdved by optical means; whilst 

(c) enabling non-damaging radiation to pass through a selected non- 
activated minor part of the shielding means to maintain restricted Anewing of the 
corresponding part of the scene. 

31. Omnidirectional ima^ng apparatus comprising first optical means having an 
annular field of view, second optical means having a fidd of view extending over a 
segment of a hemisphere and means for optically combining light recdh^ed fi'om both 
fields of view for the purpose of generating a video signal output, the first and second 
optical means bdng co-axially disposed so that their combmed fidds of view are 
hemispherical. 

32. Apparatus according to Claim 3 1, wherein the first optical means conq)rises a 
first convene reflector and a planar, concave or second convex reflector for reflecting 
light through an aperture or apertures in the first convex reflector; and wh^ein the 
second optical means con^iises an aperture or lens having the fidd of view ratending 
over the segment of the hemisphere. 
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33. Apparatus according to Claim 31, wherein the aperture or lens is centrally 
located in the first planar, concave or second convex reflector so as to direct light fi-om 
the segmental field through the aperture in the apex of the first convex reflector, the 
light from the annular field also being directed through the aperture in said apex. 

34. Apparatus according to Claim 31, wherein the lens is part of an ima^ng device 
which captures image information directly from the segmental field; the light from the 
annular field being received by another imaging device. 

35. Apparatus according to Claim 31, wherein the first optical means comprises a 
first convex reflector and a first planar, concave or second convex reflector for 
providing the annular field of view; and wherein the second optical means comprises a 
concave reflector and a third convex reflector for reflecting light from a scene to 
provide the segmental fidd of view. 

36. Apparatus according to Claim 31, inchiding an aperture or lens for directing 
light from the segmental field through the aperture in the apex of the convex reflector 
together with the light from the annular field. 

37. Apparatus according to Claim 31, wherein the first optical means conqprises a 
convex reflector and a first lens or imaging device for receiving light reflected directly 
from the convex reflector so as to provide the annular field of view, and wherein the 
second optical means comprises a second lens or imaging device having the field of 
view extending over a segment of a hemisphere. 

38. Apparatus according to Claim 31 wherein the first optical means comprises a 
convex polygonal structure including a plurahty of reflecting &cets surroimding a 
central axis and a second reflector opposite said convex reflector for reflecting Ught 
through respective apertures in said &cets ,whereby part of the scene in the annular 
field of view is reflected by the corresponding &cet and is incident on image sensing 
means coupled to the aperture; and wh^ein the second optical means comprises either a 
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lens system located in the second reflector and on said central axis^ said lens having its 
optical axis aligned ^th said central axis^ or a parabolic convex reflector and a 
second reflector opposite the convex reflector for reflecting light mto an aperture at the 
apex of the convex reflector, wherd)y light firom the segmental field of view is 
received by the or other image sensing means. 

39. Apparatus according to Claim 31 wherein the first and second optical means 
respectively comprise a convex polygonal structure including a plurality of reflecting 
fiicets surrounding a central axis and a second reflector opposite said convex reflector 
for reflecting light through respective apertures in said fiicets, whereby part of the 
scene reflected in the corresponding &cet is incident on image sensing means. 

40. Apparatus according to Claun 3 1, wherdn the convex reflector, which is part of 
the first optical means for providing the annular field of view, has an aperture at its 
apex for admitting light firom both said fields of view, the optical combining means 
being located at or below the aperture to receive the ligjht incident from both fields of 
view. 

41 . Apparatus according to Claim 31, wh^dn the second optical means comprises a 
concave reflecting surface fi>r receiving light from a scene and reflecting it on to a 
convex reflecting sur&ce which in turn reflects light into the optical combirung means. 

42. Apparatus according to Claim 3 1, wh^ein the second optical means comprises a 
convex reflecting surface for receiving light from a scene and reflecting it onto a 
concave reflecting surface which in turn reflects fight into the optical combining means. 

43. Apparatus according to Claim 2 or 42, wherein the concave reflecting surface 
and the convex reflecting surface of the second optical means are confocal parabohc 
and hyperbolic structures. 

44. Apparatus according to any of Claims 31-43, wherein the optical combining 
means comprises any one of a prismatic device, light guide means, reflecting means, or 
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any combinations thereof, for directing light from the aperture in the apex of the convex 
reflector to an imaging sur&ce of a device for converting optical images into image 
signals. 

45. Apparatus according to any of Claims 32-44, wherein the convex reflector 
which is part of the first opticat means comprises planar reflecting surfaces which are 
inclined so as to form a bases of a regular polyhedron. 

46. Apparatus according to Claim 45, wherein the planar reflecting surfaces of the 
polyhedron are sh^ed and/or dimensioned to suit the aspect ratio of an imasuag device 
on which the light from both said fields of view is incident. 

47. Apparatus according to Claim 41, wherein the first optical means comprises a 
first imaging device and convex reflector which together provide the annular field of 
view, and vtdierein the second optical means comprises a second imaging device having 
the field of view extending over a segment of a h^Doisphere. 

48. Apparatus according to any of Claims 32-45, induding means for attadung the 
apparatus to a camera to provide omnidkecdonal imaging. 

49. Apparatus according to Claim 48, wherein convex reflectors are arranged back 
to back and a laterally extending member provides the camera attacfam^. 

50. An imaging method comprising the steps of 
providing an onmidirectional unage of a scene; 
deriving image signals from said image; 

transforming the image signals into signal data having three degrees of spatial 
orientation; 

generating position related signals referenced to an artificial horizon; and 
using the position related signals to select the image signal data so as to generate 
a display of at least a part of the image of the scene, so that pitch, roll and yaw of the 
s^or is transformed into equivalent mov^ents of the displayed image. 
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51. An imagmg method according to Claim 50, wherein movement of the position 
sensor causes a relevant part of the scene to be tracked and imaged 
continuously. 

52. An ima g i ng method according to CSaim 50 or 51, in which operation is in either 
a multicast mode, where the whole of the omnidirectional image data is processed, or a 
selected zone mode, where only different parts of the scene are viewed at difiBexent 
times and the processing tracks movement aroimd the scene as different parts are 
viewed. 
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